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Hilbert spaces [TJ Def. 3.1-1] and the associated concept of orthonormal bases are of fundamental 
importance in signal processing, communications, control, and information theory. However, linear 
independence and orthonormality of the basis elements impose constraints that often make it difficult 
to have the basis elements satisfy additional desirable properties. This calls for a theory of signal 
decompositions that is flexible enough to accommodate decompositions into possibly nonorthogonal 
and redundant signal sets. The theory of frames provides such a tool. 

This chapter is an introduction to the theory of frames, which was developed by Duffin and 
Schaeffer [2] and popularized mostly through [3j-[6] . Meanwhile frame theory, in particular the aspect of 
redundancy in signal expansions, has found numerous applications such as, e.g., denoising [7j[8], code 
division multiple access (CDMA) [jj], orthogonal frequency division multiplexing (OFDM) systems 
coding theory 11, 12], quantum information theory |13|, analog-to-digital (A/D) converters 



14 
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and compressive sensing 17-19 . A more extensive list of relevant references can be found in (20f For 



a comprehensive treatment of frame theory we refer to the excellent textbook [21] . 

1 Examples of Signal Expansions 

We start by considering some simple motivating examples. 

Example 1.1 (Orthonormal basis in M. 2 ). Consider the orthonormal basis (ONB) 





T 




0" 


ei = 





e 2 = 


1 



in M 2 (see Figure 0. We can represent every signal x £ I 2 as the following linear combination of the 
basis vectors ei and e 2 : 

x = (x,ei)ei + (x,e 2 )e 2 . (1) 
To rewrite ([!]) in vector-matrix notation, we start by defining the vector of expansion coefficients as 



It is convenient to define the matrix 
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Figure 1: Orthonormal basis in 



Henceforth we call T the analysis matrix; it multiplies the signal x to produce the expansion coefficients 

c = Tx. 

Following ([!]), we can reconstruct the signal x from the coefficient vector c according to 



x = T T c = [ei e 2 ] c = [ei e 2 ] 



(x,ei) 
(x,e 2 ) 



(x, ei)ei + (x, e 2 )e 2 . 



We call 



T T = [ei e 2 ] 



1 
1 



(2) 



(3) 



the synthesis matrix; it multiplies the coefficient vector c to recover the signal x. It follows from ^ 
that is equivalent to 

x = T T Tx 



"1 0" 




"1 0" 


1 




1 



X. 



(4) 



The introduction of the analysis and the synthesis matrix in the example above may seem artificial 
and may appear as complicating matters unnecessarily. After all, both T and T T are equal to the 
identity matrix in this example. We will, however, see shortly that this notation paves the way to 
developing a unified framework for nonorthogonal and redundant signal expansions. Let us now look 
at a somewhat more interesting example. 

Example 1.2 (Biorthonormal bases in M 2 ). Consider two noncollinear unit norm vectors in M?. For 
concreteness, take (see Figure [2]) 



ei 



e-2 



V2 



For an arbitrary signal x 6 I 2 , we can compute the expansion coefficients 

c\ = (x,ei) 
c 2 = (x,e 2 ) . 

As in Example |1.1| above, we stack the expansion coefficients into a vector so that 



(x,ei)' 
(x,e 2 ) 



x 



1 

1/V2 1A/2 



x. 



(5) 
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Figure 2: Biorthonormal bases in M 2 . 
Analogously to Example we can define the analysis matrix 



[e7" 
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el 




l/y/2 1/V2. 




c = 


Tx. 



and rewrite ^ as 

Now, obviously, the vectors ei and e 2 are not orthonormal (or, equivalently, T is not unitary) so that 
we cannot write x in the form Q. We could, however, try to find a decomposition of x of the form 

x = (x,ei)ei + (x,e 2 )e 2 (6) 

with ei,§2 G M 2 . That this is, indeed, possible is easily seen by rewriting ^ according to 

x = [ei e 2 ] Tx (7) 

and choosing the vectors ei and e 2 to be given by the columns of T _1 according to 

[ei e 2 ] = T" 1 . (8) 

Note that T is invertible as a consequence of ei and e 2 not being collinear. For the specific example 
at hand we find 

[Si e 2 ] = T- 1 " ' ° 



-1 y/2 



and therefore (see Figure [2]) 



Note that mh implies that T [ei e 2 ] = I 2 , which is equivalent to 





" 1 " 




" " 


ei = 


-1 


e 2 = 


V2 



[ei e 2 ] 



"■2- 



More directly the two sets of vectors {ei,e 2 } and {ei,e 2 } satisfy a "biorthonormality" property 
according to 

'l, j = k 
0, else 



( e j,ek) 



3, k = 1,2. 



3 



1-- 



g2 




Figure 3: Overcomplete set of vectors in 



We say that {ei,e2} and {ei,e 2 } are biorthonormal bases. Analogously to ([3]), we can now define the 
synthesis matrix as follows: 

VT A 



T 1 * [ei e 2 ] = 
Our observations can be summarized according to 

(x,ei)ei + (x,e 2 )e 2 



1 
-1 y/2 



x 



T T c 

" 1 



T T Tx 



" 
V2 



1 

1/V2 l/y/2_ 





"1 0" 


X = 


1 



(9) 



Comparing ([9]) to Q, we observe the following: To synthesize x from the expansion coefficients c 
corresponding to the nonorthogonal set {ei,e2}, we need to use the synthesis matrix T T obtained 
from the set {ei,e 2 }, which forms a biorthonormal pair with {ei,e 2 }. In Example {ei, e 2 } is an 
ONB and hence T = T, or, equivalently, {ei,e 2 } forms a biorthonormal pair with itself. 

As the vectors ei and e 2 are linearly independent, the 2x2 analysis matrix T has full rank and is 
hence invertible, i.e., there is a unique matrix T _1 that satisfies T X T = I 2 . According to ([7]) this 
means that for each analysis set {ei,e 2 } there is precisely one synthesis set {ei,e 2 } such that ^ is 
satisfied for all x£i 2 . 

So far we considered nonredundant signal expansions where the number of expansion coefficients is 
equal to the dimension of the Hilbert space. Often, however, redundancy in the expansion is desirable. 

Example 1.3 (Overcomplete expansion in M 2 , |20| Ex. 3.1]). Consider the following three vectors in 
M 2 (see Figure[3}: 
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"0" 




" 1 " 


gl = 





, g2 = 


1 


, g3 = 


-1 



Three vectors in a two-dimensional space are always linearly dependent. In particular, in this example 
we have g3 = gi — g 2 . Let us compute the expansion coefficients c corresponding to {gi, g 2 , g3}: 





"ci" 




r<x, 


gi)" 




\eT 




'1 
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c = 


C2 


_A 


(x, 


ga) 




eI 


x = 
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_ c 3_ 
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g3>. 




.g 3 T . 
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X. 



(10) 
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Following Examples 1 1 . 1 1 and 1.2 we define the analysis matrix 
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and rewrite (10) as 



Tx. 



Note that here, unlike in Examples 1 1 . 1 1 and 1.2, c is a redundant representation of x as we have three 
expansion coefficients for a too-dimensional signal x. 

We next ask if x can be represented as a linear combination of the form 



X = (X,gl) gl + (X^g2) g 2 + (X^g3)g 3 

Cl C2 C3 



(11) 



with gi,g2,g3 £ IR 2 ? To answer this question (in the affirmative) we first note that the vectors gi,g2 
form an ONB for M. 2 . We therefore know that the following is true: 



Setting 



x = (x,gi)gi + (x,g 2 )g 2 . 

gl = gl, g2 = g2, g3 = 



(12) 



obviously yields a representation of the form (11). It turns out, however, that this representation is 
not unique and that an alternative representation of the form (11) can be obtained as follows. We 
start by adding zero to the right-hand side of (12): 

X = (X, gl) gl + (X, g 2 ) g 2 + (X, gl - g 2 ) (gl - gl) . 



Rearranging terms in this expression, we obtain 

x = (x, gl) 2gi + (x, g 2 ) (g 2 - gi) - (x, gl - g 2 ) gl. 
We recognize that gi — g2 = g3 and set 

gl = 2gi, g 2 = g 2 - gl, g3 = "gl- 



This allows us to rewrite ( 13 ) as 

X = (X, gl) gl + (X, g 2 ) g 2 + (X, g 3 ) g 3 . 



(13) 
(14) 



The redundant set of vectors {gi,g2,g3} is called a frame. The set {gi,g2,g3} in (14) is called a 
dual frame to the frame {gi,g2,g3}- Obviously another dual frame is given by gi = gi, g2 = g2, 
and g 3 = 0. In fact, there are infinitely many dual frames. To see this, we first define the synthesis 
matrix corresponding to a dual frame {gi, g2, g 3 } as 



T T = [gi g 2 g 3 ] • 



(15) 



It then follows that we can write 



x = (x, gi) gi + (x, g 2 ) g 2 + (x, g 3 ) g 3 
= T T c = T T Tx, 
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which implies that setting T T = [gi g 2 g3] to be a left-inverse of T yields a valid dual frame. Since T 
is a 3 x 2 ("tall") matrix, its left-inverse is not unique. In fact, T has infinitely many left-inverses (two 



of them were found above). Every left-inverse of T leads to a dual frame according to (15). 



Thanks to the redundancy of the frame {gi, g 2 , g3}, we obtain design freedom: In order to synthesize 
the signal x from its expansion coefficients c k = (x, g^) , k = 1, 2, 3, in the frame {gi, g2, g3}, we can 
choose between infinitely many dual frames {gi, g 2 , g3j-. In practice the particular choice of the dual 
frame is usually dictated by the requirements of the specific problem at hand. We shall discuss this 



issue in detail in the context of sampling theory in Section 4.2 



2 Signal Expansions in Finite-Dimensional Hilbert Spaces 

Motivated by the examples above, we now consider general signal expansions in finite-dimensional 
Hilbert spaces. As in the previous section, we first review the concept of an ONB, we then consider 
arbitrary (nonorthogonal) bases, and, finally, we discuss redundant vector sets — frames. While the 
discussion in this section is confined to the finite-dimensional case, we develop the general (possibly 
infinite-dimensional) case in Section [3| 

2.1 Orthonormal Bases 

We start by reviewing the concept of an ONB. 

Definition 2.1. The set of vectors {e k }^ =1 , e k € C M , is called an ONB for C M if 

1. span{e fc }^l 1 = {ciei + c 2 e 2 + . . . + c M eu | ci, c 2 , . . . , cm £ 
2. 

(e k ,ej) = { ; / k,j = 1, . . . ,M. 




When {e k } k Li is an ONB, thanks to the spanning property in Definition 2.1, every x € C M can 



•>M 



be decomposed as 



M 



(16) 



k=l 



The expansion coefficients {c k }^ =1 in (16) can be found through the following calculation: 

I M \ M 



(x, ej) = ( c k e k , ej\ = ) j c k {e k ,ej} = Cj. 



\k=l 



k=l 



In summary, we have the decomposition 



M 



x = ^ (x, e fc ) e k 



k=l 



Just like in Example |1.1[ in the previous section, we define the analysis matrix 



-M. 
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If we organize the inner products {(x, G k )}¥^ into the vector c, we have 



(x,ei) 


= Tx = 


rp H i 


_(x,e M )_ 







x. 



Thanks to the orthonormality of the vectors e%,e2, . ■ ■ , &m the matrix T is unitary, i.e., T H = T 1 
and hence 



TT 



-AI. 



ei ... e M 



(ei,ei) ••• (e A /,ei) 



(ei,ej\/) ••• (e M ,e M } 



*-M 



Thus, if we multiply the vector c by T H , we synthesize x according to 



M 



T H c = T H Tx = (x, e k ) e k = I M x = x. 



(17) 



fc=l 



We shall therefore call the matrix T H the synthesis matrix, corresponding to the analysis matrix T. 
In the ONB case considered here the synthesis matrix is simply the Hermitian adjoint of the analysis 
matrix. 



2.2 General Bases 



We next relax the orthonormality property, i.e., the second condition in Definition 2.1, and consider 
general bases. 

Definition 2.2. The set of vectors {e k } k=1 , e k G C M , is a basis for C M if 

1. span{e fc }^l 1 = {ciei + c 2 e 2 + . . . + c M e M |ci,c 2) . . . ,c M 6 C} = C A/ 

2. {e k }ff_ 1 is a linearly independent set, i.e., if ^fcli c k e k = for some scalar coefficients {c k }^ =l , 
then necessarily c k = for all k = 1, ... , M. 

Now consider a signal x G C M and compute the expansion coefficients 

c fe ^(x,e fc ), k = l,...,M. (18) 

Again, it is convenient to introduce the analysis matrix 



-Ml 



and to stack the coefficients {c k }^ =1 in the vector c. Then (18) can be written as 



c = Tx. 

Next, let us ask how we can find a set of vectors {ei, . . . , ejvf}, &k £ C M , k = 1, . . . , M, that is dual to 
the set {ei, . . . , e^/} in the sense that 



M 



M 



x = ^ c k e k = ^ (x, ejfe) e k 



(19) 



k=i 



k=l 
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for all x € C M . If we introduce the synthesis matrix 



t H 4 [ex ••• e M ], 



we can rewrite ( 19 ) in vector-matrix notation as follows 



x = f H c = f H Tx. 



(3m that satisfy ( 19 ) is equivalent to finding the inverse of the 

-l 



This shows that finding vectors ei, . 

analysis matrix T and setting T H = T _1 . Thanks to the linear independence of the vectors {^k}M=v 
the matrix T has full rank and is, therefore, invertible. 

Summarizing our findings, we conclude that in the case of a basis {efc}^L 1 , the analysis matrix and 
the synthesis matrix are inverses of each other, i.e., T H T = TT H = Im- Recall that in the case of an 
ONB the analysis matrix T is unitary and hence its inverse is simply given by T H [see 07M, so that 
in this case T = T. 

Next, note that TT H = Im is equivalent to 



J M. 



ei ... e M 



(ei,ei) 



(eA/,ei) 



(ei,e M ) ••• (eM,e M ) 



or equivalently 




k,j = 1,...,M. 



(20) 



M 



k=i and {efc}fc=i 



are biorthonormal bases. ONBs are biorthonormal to themselves in 



The sets {e^} 

this terminology, as already noted in Example 1.2 We emphasize that it is the fact that T and T H 
are square and full-rank that allows us to conclude that T H T = Im implies TT H = Im and hence to 
conclude that |20[ ) holds. We shall see below that for redundant expansions T is a tall matrix and 
T H T TT H (T H T and TT H have different dimensions) so that dual frames will not be biorthonormal. 
As T is a square matrix and of full rank, its inverse is unique, which means that for a given analysis 



set {e k } 



M 



the synthesis set {e/ c }^£ 1 is unique. 



Alternatively, for a given synthesis set {ek}^ =1 , there 



\ fc = \, tixc oy iitiicDiD oct ^cfcj^—^ id (iutyuc. nuciiianvcij, ivji a given ojiitncoio oci , 

is a unique analysis set {B] c }uL 1 . This uniqueness property is not always desirable. For example, one 
may want to impose certain structural properties on the synthesis set {efc}^£ 1 in which case having 
freedom in choosing the synthesis set as in Example 1.2 is helpful. 



An important property of ONBs is that they are norm-preserving: The norm of the coefficient 
vector c is equal to the norm of the signal x. This can easily be seen by noting that 



c H c 



x H T H Tx 



x H I M x 



X 



(21) 



where we used (17). Biorthonormal bases are not norm-preserving, in general. Rather, the equality in 



(21) is relaxed to a double-inequality, by application of the Rayleigh-Ritz theorem 22, Sec. 9.7.2.2] 
according to 



At, 



rj-iH rj-i 



Ix|l 2 < 



x H T H Tx < A„ 



; (T H T) ||x|| 



(22) 



2.3 Redundant Signal Expansions 

The signal expansions we considered so far are non-redundant in the sense that the number of 
expansion coefficients equals the dimension of the Hilbert space. Such signal expansions have a 
number of disadvantages. First, corruption or loss of expansion coefficients can result in significant 
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reconstruction errors. Second, the reconstruction process is very rigid: As we have seen in Section [2. 2[ 
for each set of analysis vectors, there is a unique set of synthesis vectors. In practical applications it is 
often desirable to impose additional constraints on the reconstruction functions, such as smoothness 
properties or structural properties that allow for computationally efficient reconstruction. 

Redundant expansions allow to overcome many of these problems as they offer design freedom and 



robustness to corruption or loss of expansion coefficients. We already saw in Example 1.3 that in the 
case of redundant expansions, for a given set of analysis vectors the set of synthesis vectors that allows 
perfect recovery of a signal from its expansion coefficients is not unique; in fact, there are infinitely 
many sets of synthesis vectors, in general. This results in design freedom and provides robustness. 
Suppose that the expansion coefficient C3 = (x, g3) in Example 1.3 is corrupted or even completely 



lost. We can still reconstruct x exactly from (12). 



Now, let us turn to developing the general theory of redundant signal expansions in finite-dimensional 
Hilbert spaces. Consider a set of N vectors {gi, . . . , gAr}, gk £ C M , k = 1, . . . , N, with TV" > M. Clearly, 
when N is strictly greater than M, the vectors gi, . . . , gjv must be linearly dependent. Next, consider 
a signal x E C M and compute the expansion coefficients 



Cfc = (x,g fc ), k = l,...,N. 
Just as before, it is convenient to introduce the analysis matrix 

gi 



(23) 



(24) 



and to stack the coefficients {c/ c }^ =1 in the vector c. Then (23) can be written as 



Tx. 



(25) 



Note that c G and x E C 1U . Differently from ONBs and biorthonormal bases considered in 



Sections 2.1 and 2.2, respectively, in the case of redundant expansions, the signal x and the expansion 
coefficient vector c will, in general, belong to different Hilbert spaces. 

The question now is: How can we find a set of vectors {gi, . . . , gjv}, gk G C M , k = 1, . . . , N, such 
that 

N N 

x = ^2 Ck % k = X] ( x ' gfc ^ & k 

k=l k=l 

for all x G C M ? If we introduce the synthesis matrix 



(26) 



t H 4 [ gl 



we can rewrite (26) in vector-matrix notation as follows 



x = T H c = T H Tx. 



(27) 



Finding vectors gi , . 
T H of T, i.e., 



gTV that satisfy ( 26 ) for all x G C M is therefore equivalent to finding a left-inverse 



rj-iH r£i 



First note that T is left-invertible if and only if C 



M _ 



span{gjt}^; =1 , i.e., if and only if the set of 
vectors {gk\k=i s P ans C M . Next observe that when N > M, the N x M matrix T is a "tall" matrix, 



and therefore its left-inverse is, in general, not unique. In fact, there are infinitely many left-inverses. 



The following theorem 23 Ch. 2, Th. 1] provides a convenient parametrization of all these left-inverses. 
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Theorem 2.3. Let A G C NxM , N > M. Assume that rank(A) = M. Then At = (A H A) -1 A H is a 
left-inverse of A, i.e., At A = Im- Moreover, the general solution L G <C MxN of the equation LA = Im 
is given by 

L = A f + M(Ijv - AAt), (28) 
where M G £, MxN is an arbitrary matrix. 

Proof. Since rank(A) = M, the matrix A H A is invertible and hence At is well denned. Now, let us 
verify that At is, indeed, a left-inverse of A: 

At A = (A H A)- 1 A H A = I M - (29) 

The matrix At is called the Moore- Penrose inverse of A. 

Next, we show that every matrix L of the form (28) is a valid left-inverse of A: 

LA = ^A f + M(Lv - AA f )) A 

= AtA +MA - MA A+A 

Im Im 
= I M + MA-MA = I M , 



where we used ( 29 ) twice. 

Finally, assume that L is a valid left-inverse of A, i.e., L is a solution of the equation LA = Im- 
We show that L can be written in the form (28). Multiplying the equation LA = Im by At from the 
right, we have 

LA At = At. 

Adding L to both sides of this equation and rearranging terms yields 

L = At + L - LAAt = At + L(I N - AAt), 



which shows that L can be written in the form (28) (with M = L), as required. □ 

We conclude that for each redundant set of vectors {gi, . . . , g^v} that spans C A/ , there are infinitely 
many dual sets {gi, . . . , g^v} such that the decomposition (26) holds for all x G C M . These dual sets 
are obtained by identifying {gi, . . . , g^v} with the columns of L according to 

[gi • • • gjv] = L, 

where L can be written as follows 

L = T f + M(Lv - TT f ) 

and M G <C MxN is an arbitrary matrix. 

The dual set {gi, . . . ,gjv} corresponding to the Moore-Penrose inverse L = Tt of the matrix T, 

i.e., 

[gi • • • sn] = Tt = (T H T)" 1 T H 



is called the canonical dual of {gi, . . . ,gjv}. Using (24), we see that in this case 

gfe = (T H T) -1 gfc, fc = l,...,JV. (30) 

Note that unlike in the case of a basis, the equation T H T = Im does not imply that the sets {gk}k=i 
and {gfcjj^Li are biorthonormal. This is because the matrix T is not a square matrix, and thus, 
T H T / TT H (f H T and TT H have different dimensions). 
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Similar to biorthonormal bases, redundant sets of vectors are, in general, not norm-preserving. 



Indeed, from ( 25 ) we see that 

||c|| 2 = x H T H Tx 

and thus, by the Rayleigh-Ritz theorem |22| Sec. 9.7.2.2], we have 

^i. r ,\ , — 11=2 < || c ||2 < Amax ^ T H X ^j || x ||2 (31) 

as in the case of biorthonormal bases. 

We already saw some of the basic issues that a theory of orthonormal, biorthonormal, and redundant 
signal expansions should address. It should account for the signals and the expansion coefficients 
belonging, potentially, to different Hilbert spaces; it should account for the fact that for a given analysis 
set, the synthesis set is not unique in the redundant case, it should prescribe how synthesis vectors can 
be obtained from the analysis vectors. Finally, it should apply not only to finite-dimensional Hilbert 
spaces, as considered so far, but also to infinite-dimensional Hilbert spaces. We now proceed to develop 
this general theory, known as the theory of frames. 



3 Frames for General Hilbert Spaces 

Let {gk}keK is a countable set) be a set of elements taken from the Hilbert space %. Note that 
this set need not be orthogonal. 

In developing a general theory of signal expansions in Hilbert spaces, as outlined at the end of the 
previous section, we start by noting that the central quantity in Section [2] was the analysis matrix T 
associated to the (possibly nonorthogonal or redundant) set of vectors {gi, . . . , gjv}. Now matrices 
are nothing but linear operators in finite-dimensional Hilbert spaces. In formulating frame theory for 
general (possibly infinite-dimensional) Hilbert spaces, it is therefore sensible to define the analysis 
operator T that assigns to each signal x E H the sequence of inner products Tx = {(x, gk)}ke/c- 
Throughout this section, we assume that {gk}keK is a Bessel sequence, i.e., X^fceA: \{ x i9h)\ 2 < °o for 
all x € H. 

Definition 3.1. The linear operator T is defined as the operator that maps the Hilbert space % into 
the space I 2 of square-summable complex sequence^} T : % — > I 2 , by assigning to each signal x € T~L 
the sequence of inner products (x,gk) according to 

T : x -> {(x,g k }}keic. 

Note that ||Tx|| 2 = X^fceA: \ ( x ^9k)\ 2 ^ i- e -j the energy ||Tx|| 2 of Tx can be expressed as 

||Tx|| 2 = ^Kx,<7 fc )| 2 . (32) 

keK. 

We shall next formulate properties that the set {gk}keic and hence the operator T should satisfy if we 
have signal expansions in mind: 

1. The signal x can be perfectly reconstructed from the coefficients {(x,gk)}keic- This means that 
we want (x,gk) = {y,gk), for all k G JC, (i.e., Tx = Ty) to imply that x = y, for all x,y € H. 
In other words, the operator T has to be left-invertible, which means that T is invertible on its 
range space 1Z(T) = {y G I 2 : y = Tx, x £ %}. 

lr The fact that the range space of T is contained in I 2 is a consequence of {gk}keK being a Bessel sequence. 
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This requirement will clearly be satisfied if we demand that there exist a constant A > such 
that for all x, y G % we have 

A\\x-y\\ 2 < ||Tx-Ty|| 2 . 
Setting z = x — y and using the linearity of T, we see that this condition is equivalent to 

A\\zf < \\Tz\\ 2 (33) 

for all z G % with A > 0. 

2. The energy in the sequence of expansion coefficients Tx = {(x, gk)}keK should be related to the 
energy in the signal x. For example, we saw in (21) that if {ejt}^£ 1 is an ONB for C M , then 



M 



|Tx|| 2 = ^|(x,e fc )| 2 = ||x|| 2 , for all x G 



(34) 



k=l 



This property is a consequence of the unitarity of T = T and it is clear that it will not hold for 



general sets {g k }keK. (see t ne discussion around (22) and (|3T|) ) . Instead, we will relax (34) to 
demand that for all x G % there exist a finite constant B such tha10 



\Tx\\ 2 = Y,\(^9k)\ 2 <B\ 



x 



(35) 



keK. 



Together with (33) this "sandwiches" the quantity ||Tx|| 2 according to 

y4||x|| 2 < ||Tx|| 2 < B\\x\\ 2 . 
We are now ready to formally define a frame for the Hilbert space T~L. 

Definition 3.2. A set of elements {gk}keic, 9k G H, k G /C, is called a frame for the Hilbert space % if 

A\\x\\ 2 <^\{x,g k )\ 2 <B\\x\\ 2 , for all xeV., (36) 

keK. 

with A, B G M and < A < B < oo. Valid constants A and B are called frame bounds. The largest 
valid constant A and the smallest valid constant B are called the (tightest possible) frame bounds. 

Let us next consider some simple examples of frames. 



Example 3.3 ([21]). Let {efc}^ =1 be an ONB for an infinite-dimensional Hilbert space H. By repeating 
each element in {ek} < k L 1 once, we obtain the redundant set 

{fffcjfcli = {ei,ei,e 2 ,e 2 , . . .}• 
To see that this set is a frame for H, we note that because {ek}^ =1 is an ONB, for all x G 7i, we have 

oo 

^2\(x,e k )\ 2 = \\x\\ 2 



k=i 



and therefore 



fc=l 



fc=l 



fc=l 



This verifies the frame condition (36) and shows that the frame bounds are given by A = B = 2. 



"Note that if (351 is satisfied with B < oo, then {gk}keic is a Bessel sequence. 
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Example 3.4 (|21|). Starting from the ONB {ek} < ^ =1 , we can construct another redundant set as 
follows 

r ,00 J 1 1 1 1 1 1 

= (ei, ^e 2 , ^e 3 , -^3, . . . j . 

To see that the set {gk}k^=i ls a f rame f° r take an arbitrary x £ H and note that 

00 oo/^>2oo^ 00 



k 

k=l k=l 



k=l k=l 

We conclude that {gk}i*Li is a & ame with the frame bounds A = B = 1 



From (32) it follows that an equivalent formulation of (36) is 



A\\x\\ 2 < \\Tx\\ 2 < B\\x\\ 2 , for all xeH. 

This means that the energy in the coefficient sequence Tx is bounded above and below by bounds 
that are proportional to the signal energy. The existence of a lower frame bound A > guarantees 
that the linear operator T is left-invertible, i.e., our first requirement above is satisfied. Besides that it 
also guarantees completeness of the set {gk}keK for H, as we shall see next. To this end, we first recall 
the following definition: 

Definition 3.5. A set of elements {gk}k&c, gk G H, k G K., is complete for the Hilbert space H 
if {x,gk) = for all k € K, and with x G H implies x = 0, i.e., the only element in Ti that is orthogonal 
to all gk, is x = 0. 

To see that the frame {gk}k£ic is complete for T~L, take an arbitrary signal x G % and assume 
that {x,gk) = for all k G /C. Due to the existence of a lower frame bound A > we have 

^IM| 2 <^|(x, 5fc >| 2 = o, 

which implies ||x|| 2 = and hence x = 0. 

Finally, note that the existence of an upper frame bound B < 00 guarantees that T is a bounded 
linear operatoij^] (see [TJ Def. 2.7-1]), and, therefore (see [lj Th. 2.7-9]), continuous^] (see [TJ Sec. 2.7]). 

Recall that we would like to find a general method to reconstruct a signal x G H from its expansion 



coefficients {(x, gk)}keic- In Section 2.3 we saw that in the finite-dimensional case, this can be 
accomplished according to 

N 

x= ^(x,g fc )g fc . 

k=l 

Here {gi, . . . , g^v} can be chosen to be the canonical dual set to the set {gi, . . . , gAr} and can be 
computed as follows: gt = (T H T) _1 gfc, k = 1, . . . , N. We already know that T is the generalization 
of T to the infinite-dimensional setting. Which operator will then correspond to T H ? To answer this 
question we start with a definition. 



3 Let H and H' be Hilbert spaces and A : T-L — > W a linear operator. The operator A is said to be bounded if there 
exists a finite number c such that for all x £ H, \\Ax\\ < c\\x\\. 

4 Let H and %' be Hilbert spaces and A : H — > H' a linear operator. The operator A is said to be continuous at a point 
xo £ H if for every e > there is a S > such that for all x £ H satisfying \\x — aro ] | < 5 it follows that ||Aa; — Axo|| < £• 
The operator A is said to be continuous onH, if it is continuous at every point xq £ H. 
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Definition 3.6. The linear operator T x is denned as 

T x : I 2 -> U 

T x : {c k } kelc -> £ 
keK. 

Next, we recall the definition of the adjoint of an operator. 

Definition 3.7. Let A : T~L — > %' be a bounded linear operator between the Hilbert spaces % and %' . 
The unique bounded linear operator A* : H! — > H that satisfies 

(Ax,y) = (x,A*y) (37) 

for all x G % and all y G is called the adjoint of A. 

Note that the concept of the adjoint of an operator directly generalizes that of the Hermitian 
transpose of a matrix: if A E C NxM , x G C M , y G C , then 

(Ax, y) = y H Ax = (A H y) H x = (x, A H y 



which, comparing to (37), shows that A H corresponds to A*. 

We shall next show that the operator T x defined above is nothing but the adjoint T* of the 
operator T. To see this consider an arbitrary sequence {ck}ketc G I 2 and an arbitrary signal x G 7i- 
We have to prove that 

(Tx,{c k } k( z K ) = (x,T x {c k } keK ) ■ 
This can be established by noting that 

(Tx, {c fe } fc& »c) = ^2 (x, g k ) c* k 

x, ^2 Ck 9k ) = ^2 c k ( x > 9k) ■ 

keK. I keK. 

We therefore showed that the adjoint operator of T is T x , i.e., 

In what follows, we shall always write T* instead of T x . As pointed out above the concept of the adjoint 
of an operator generalizes the concept of the Hermitian transpose of a matrix to the infinite-dimensional 
case. Thus, T* is the generalization of T H to the infinite-dimensional setting. 

3.1 The Frame Operator 




Let us return to the discussion we had immediately before Definition 3.6 We saw that in the finite- 
dimensional case, the canonical dual set {gi, . . . ,g7v} to the set {gi, . . . ,gAr} can be computed as 
follows: gfc = (T H T) -1 gfc, k = 1, . . . , N. We know that T is the generalization of T to the infinite- 
dimensional case and we have just seen that T* is the generalization of T H . It is now obvious that the 
operator T*T must correspond to T H T. The operator T*T is of central importance in frame theory. 

Definition 3.8. Let {g k }k£ic be a frame for the Hilbert space T~L. The operator S : T-L — > % defined as 

S = T*T, (38) 

§x = ^2 ( x i9k)9k 

k£lC 

is called the frame operator. 
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We note that 

^2\(x,g k )\ 2 = \\Txf = (Tx, Tx) = (T*Tx,x) = (Sx, x) . (39) 

k&K 

We are now able to formulate the frame condition in terms of the frame operator § by simply noting 



that ( 36 ) can be written as 

A\\x\\ 2 <(Sx,x) <B\\x\\ 2 . (40) 
We shall next discuss the properties of §. 

Theorem 3.9. The frame operator S satisfies the properties: 

1. § is linear and bounded; 

2. S is self-adjoint, i.e., S* = S; 

3. S is positive definite, i.e., (Sx, x) > for all x E H; 

4- § has a unique self-adjoint positive definite square root (denoted as S 1 ^ 2 ). 

Proof. 1. Linearity and boundedness of S follow from the fact that § is obtained by cascading a 
bounded linear operator and its adjoint (see fl38|)). 



2. To see that § is self- adjoint simply note that 

S* = Hp TV = T*T = S. 



3. To see that S is positive definite note that, with (40) 

(§x,x) > A\\x\\ 2 > 

for all x E 7i, 0. 

4. Recall the following basic fact from functional analysis 111 Th. 9.4-2]. 

Lemma 3.10. Every self-adjoint positive definite bounded operator A : % — > % has a unique 
self- adjoint positive definite square root, i.e., there exists a unique self-adjoint positive-definite 
operator B such that A = BB. The operator B commutes with the operator A, i.e., BA = AB. 



Property [4] now follows directly form Property [2j Property [3j and Lemma 3.10 



□ 

We next show that the tightest possible frame bounds A and B are given by the smallest and the 
largest spectral value |1, Def. 7.2-1] of the frame operator §, respectively. 

Theorem 3.11. Let A and B be the tightest possible frame bounds for a frame with frame operator S. 
Then 

A = A min and B = A max , (41) 
where A m i n and A max denote the smallest and the largest spectral value of S, respectively. 
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Proof. By standard results on the spectrum of self-adjoint operators [Tl Th. 9.2-1, Th. 9.2-3, Th. 
9.2-4], we have 



An 



inf 



(Sx, x) 



and A r 



sup 



(Sx, x) 



xen \\x\\* 1 1 a? 

This means that A m i n and A max are, respectively, the largest and the smallest constants such that 



(42) 



A m i n 1 1 ^ 1 1 2 — (Sx,x) < A r 



(43) 



is satisfied for every x € %. According to (40) this implies that A m i n and A max are the tightest possible 
frame bounds. □ 



It is instructive to compare (43) to (31). Remember that § = T*T corresponds to the matrix T H T 

Thus, ||c|| 2 = x H T H Tx = (Sx,x), which upon 



in the finite-dimensional case considered in Section 



2.3 



insertion into (31), shows that (43) is simply a generalization of (31) to the infinite-dimensional case. 



3.2 The Canonical Dual Frame 



Recall that in the finite-dimensional case considered in Section 2.3 the canonical dual frame {gk}k=i °f 
the frame {gk}k=i can °e used to reconstruct the signal x from the expansion coefficients {(x, gk)}k=\ 
according to 



N 

x = J^(x,gfc)gfc. 

fc=l 



In (30) we saw that the canonical dual frame can be computed as follows: 

gfc = (T H T) -1 gfc, k = l,...,N. 



(44) 



We already pointed out that the frame operator S = T*T is represented by the matrix T H T in the 
finite-dimensional case. The matrix (T H T) _1 therefore corresponds to the operator S -1 , which will be 
studied next. 



From (41) it follows that A m i n , the smallest spectral value of S, satisfies A m j n > if {gk}keic is a 



frame. This implies that zero is a regular value [T] Def. 7.2-1] of S and hence S is invertible on 7i, i.e., 
there exists a unique operator S _1 such that SS _1 = S _1 S = 1%. Next, we summarize the properties 
of S _1 . 

Theorem 3.12. The following properties hold: 

1. is self-adjoint, i.e., (S -1 )* 

2. S _1 satisfies 



2-1 • 



1 

B 



inf 



and 



1 

A 



sup 



(S 1 x,x) 



(45) 



where A and B are the tightest possible frame bounds of S; 
3. is positive definite. 
Proof. 1. To prove that § _1 is self-adjoint we write 

(SS- 1 )* = (S- 1 )*S* = l H . 
Since § is self-adjoint, i.e., S = §*, we conclude that 

(s- 1 )*s = i«. 
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Multiplying by § 1 from the right, we finally obtain 



2. To prove the first equation in (45) we write 

(§x,x) (SSVZs-iy.S 1 ^-^) 



B 



sup 



len <S 1 /2s-i 2/) Si/2 S -iy) 



sup 



( § -l§l/2g § l/2 § -l y); 



(46) 



where the first equality follows from (41) and (42); in the second equality we used the fact that 
the operator S 1//2 S _1 is one-to-one on % and changed variables according to x 



SV2S-1 



y; m 



the third equality we used the fact that S 1 / 2 and § 1 are self-adjoint, and in the fourth equality 
we used § = S 1 / 2 ^ 1 / 2 . The first equation in (45) is now obtained by noting that (46) implies 




sup , 

y &i (a y,y) 



(y,y) 



inf 

yen 



(y,y) 



The second equation in ( 45 ) is proved analogously. 



3. Positive-definiteness of S 1 follows from the first equation in (45) and the fact that B < oo so 
that 1/B > 0. 

□ 



We are now ready to generalize ( 44 ) and state the main result on canonical dual frames in the case 



of general (possibly infinite-dimensional) Hilbert spaces. 

Theorem 3.13. Let {gk}ketc be a frame for the Hilbert space % with the frame bounds A and B, and 
let S be the corresponding frame operator. Then, the set {gk}kefc given by 



gk 



S- 1 ^, keJC, 



(47) 



is a frame for % with the frame bounds A = 1/B and B = 1/A. 
The analysis operator associated to {{jk}keK defined as 

t :H^l 2 



T : x -)■ {(x,gk)}keic 



satisfies 



f = T§ 1 = T(T*T) _1 . 
Proof. Recall that §> _1 is self-adjoint. Hence, we have (x, gk} 



Thus, using (39), we obtain 



(48) 

= (S~ 1 x,gk) for all x £ ~H. 



ke/c 



keic 



§ l x,9k)\ 

(S(S~ 1 x),S~ 1 x) = (x^x) = <S _1 x,x> . 



Therefore, we conclude from (45) that 

1 „ 

jg\\* 



< Yl \( x '9k)\ 2 < ^Ikll 2 , 



ketc 
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i.e., the set {g k } k £ic constitutes a frame for 7-L with frame bounds A = 1/B and B = 1/A; moreover, it 



follows from (45) that A = 1/B and B = 1/A are the tightest possible frame bounds. It remains to 
show that f = 

fx = {{x,g k )} kefC = {( x ' S ~ 1 5fc>}jfc e x: = {( S ~ lx >9k)} kEK = TS^x. 

□ 

We call {gk}k&c the canonical dual frame associated to the frame {g k } k £K- It is convenient to 
introduce the canonical dual frame operator: 

Definition 3.14. The frame operator associated to the canonical dual frame, 

§ = f*f, Sx = ^(x,5fc)5fe (49) 

keK. 

is called the canonical dual frame operator. 

Theorem 3.15. The canonical dual frame operator S satisfies S = S . 
Proof. For every x £ H, we have 

§x = ^2 9k) 9k = ^^ (x, §~ 1 gk) 

= S" 1 (^x, g k ) gk = § _1 §§ _1 :c = S^x, 

keK. 



where in the first equality we used ( 49 ) , in the second we used ( 47 ) , in the third we made use of the 



fact that § 1 is self-adjoint, and in the fourth we used the definition of S. □ 

Note that canonical duality is a reciprocity relation. If the frame {(jk}keK, is the canonical dual 
of the frame {g k } k £ic, then {gk}kEK is the canonical dual of the frame {gk}k<aK- This can be seen by 
noting that 

S- 1 ^ = (S- 1 )- 1 ^ 1 ^ = SS^gk = g k . 

3.3 Signal Expansions 

The following theorem can be considered as one of the central results in frame theory. It states that 
every signal x £ % can be expanded into a frame. The expansion coefficients can be chosen as the 
inner products of x with the canonical dual frame elements. 

Theorem 3.16. Let {g k } k <=jc an d {^fcjfceA: be canonical dual frames for the Hilbert space H. Every 
signal x £ % can be decomposed as follows 

x = T*fx = (x, g k ) gk 

keK. 



x = f*Tx = ^ (x,g k )g k . (50) 

k&K. 

Note that, equivalently, we have 



T*T = T*T = I 



H- 
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Proof. We have 



t*t x = ^2 ( x > 9k) 9k = ^2 ( x ' s lak ) 9k 

keK keic 
= ^2 (^x, g k ) g k = §§ _1 x 



x. 



k&K. 



This proves that T*T = 1%. The proof of T*T = 1% is similar. 



□ 



Note that (50) corresponds to the decomposition (26) we found in the finite-dimensional case. 



It is now natural to ask whether reconstruction of x from the coefficients {x,g k ) , k G /C, according 



to (50) is the only way of recovering x from {x,g k ) , k G /C. Recall that we showed in the finite- 
dimensional case (see Section 2.3) that for each complete and redundant set of vectors {gi, . . . ,gw}, 
there are infinitely many dual sets {gi, . . . , gTv} that can be used to reconstruct a signal x from 
the coefficients (x, g^) , k = 1, . . . , N, according to (26). These dual sets are obtained by identifying 



{gi> ■ ■ • > giv} with the columns of L, where L is a left-inverse of the analysis matrix T. In the infinite- 
dimensional case the question of finding all dual frames for a given frame boils down to finding, for a 
given analysis operator T, all linear operators L that satisfy 

L,Tx = x 

for all x G H. In other words, we want to identify all left-inverses L of the analysis operator T. The 
answer to this question is the infinite-dimensional version of Theorem |2.3| that we state here without 
proof. 

Theorem 3.17. Let A : 7~L — > I 2 be a bounded linear operator. Assume that A* A :1-L — > 7~L is invertible 
on T-L. Then, the operator A^ : I 2 — > H defined as A^ = (A*A) _1 A* is a left-inverse of A, i.e., At A = 1%, 
where 1% is the identity operator on T~L. Moreover, the general solution L of the equation LA = 1% is 
given by 

L = A f + M(Kja - AA f ) 

where M : I 2 -> U is an arbitrary bounded linear operator and Up is the identity operator on I . 
Applying this theorem to the operator T we see that all left-inverses of T can be written as 

L = T f + M(I Z 2 - TT T ) (51) 
where M : I 2 — > % is an arbitrary bounded linear operator and 

T t = (x*T) _1 T*. 



Now, using (48), we obtain the following important identity: 

T t = (x*T) _1 T* = S^T = f *. 



This shows that reconstruction according to (50), i.e., by applying the operator T* to the coefficient 



sequence Tx = {{x, gk)}k£ic is nothing but applying the infinite-dimensional analog of the Moore- 
Penrose inverse T T = (T H T)~ 1 T H . As already noted in the finite-dimensional case the existence of 
infinitely many left-inverses of the operator T provides us with freedom in designing dual frames. 



We close this discussion with a geometric interpretation of the parametrization (51). First observe 
the following. 
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Theorem 3.18. The operator 

F:l 2 ^ K(T) C I 2 



defined as 

satisfies the following properties: 

1. P is the identity operator 'Ip on 1Z(T). 

2. P is the zero operator on TKT)^, where IZ^) 1 - denotes the orthogonal complement of the 
space 1Z(T). 

In other words, P is the orthogonal projection operator onto 1Z(T) = {{c k }keK I { c k}keK = ^x,x £ %}, 
the range space of the operator T. 

Proof. 1. Take a sequence {c k } k< z/c G 7£(T) and note that it can be written as {ck}k^K, — Tx, where 
x 6 H. Then, we have 

Hc k }keK = TS^TTx = TS^Sx = TI H x = Tx = {c k } keK . 

This proves that P is the identity operator on 1Z(T). 

2. Next, take a sequence {c k } k& ic £ "/^(T)- 1 . As the orthogonal complement of the range space of 
an operator is the null space of its adjoint, we have T*{c k } ke jc = and therefore 

V{c k } keK = TS^T* {c k }keic = 0. 
This proves that P is the zero operator on T^(T)- 1 -. 

□ 

Now using that IT* = TS^T* = P and = S^T* = S^SS^T* = S" 1 T*TS- 1 T* = f *P, we can 



rewrite (51) as follows 

L = T*P + M(Ip — P). (52) 

Next, we show that (Ip — P) : I 2 — > I 2 is the orthogonal projection onto T^(T)- 1 ". Indeed, we can directly 
verify the following: For every {c fe } fee /c G T^T)- 1 , we have (Ip -F){c k }k£K = Ip{cfe}fce/C - = {c k }keK, 
i.e., Ip — P is the identity operator on ^(T)- 1 ; for every {c k } k £tc £ (^(T)- 1 )- 1 - = TZ(T), we have 
(Ip - P){c fc } fce /c = I«2{ c fc}fc6/c - {ckjkeK. = 0, i.e., Ip - P is the zero operator on (^(T)^)- 1 . 



We are now ready to re-interpret (52) as follows. Every left-inverse L of T acts as T* (the synthesis 
operator of the canonical dual frame) on the range space of the analysis operator T, and can act in an 
arbitrary linear and bounded fashion on the orthogonal complement of the range space of the analysis 
operator T. 

3.4 Tight Frames 



The frames considered in Examples 3.3 and 3.4 above have an interesting property: In both cases the 



tightest possible frame bounds A and B are equal. Frames with this property are called tight frames. 
Definition 3.19. A frame {g k }ke)C with tightest possible frame bounds A = B is called a tight frame. 
Tight frames are of significant practical interest because of the following central fact. 
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Theorem 3.20. Let {gk}k<aic be a frame for the Hilbert space %. The frame {gk}keic is tight with 
frame bound A if and only if its corresponding frame operator satisfies § = A\%, or equivalently, if 



x 



9k) gk 



(53) 



fce/c 



for all x £ rl. 

Proof. First observe that S = A\u is equivalent to §x = Aly_x = Ax for all x £ T~L, which, in turn, is 
equivalent to (53) by definition of the frame operator. 



To prove that tightness of {gk}keK implies S = Al%, note that by Definition 3.19, using (40) we 
can write 

(§x, x) = A (x, x) , for all x € H. 



Therefore 



((§ - AI n )x, x) = 0, for all x€H, 



which implies § = A\%. 

To prove that § = AI% implies tightness of {gk}keic, we take the inner product with x on both 
sides of ( 53 ) to obtain 



( x > x ) = (9k, 



fee/c 



This is equivalent to 



A \\ x \\ 2 = ^2 \( x >9k)\ 2 i 



keK 



which shows that {gk}keK. is a tight frame for % with frame bound equal to A. 



□ 



The practical importance of tight frames lies in the fact that they make the computation of the 
canonical dual frame, which in the general case requires inversion of an operator and application of 
this inverse to all frame elements, particularly simple. Specifically, we have: 



gk = S 1 gk 



1 1 

~^H9k = ~^9k- 



A well-known example of a tight frame for R is the following: 

Example 3.21 (The Mercedes-Benz frame [20]). The Mercedes-Benz frame (see Figure [4]) is given by 
the following three vectors in M?: 



g2 



-VS/2 
-1/2 



g3 



-1/2 



(54) 



To see that this frame is indeed tight, note that its analysis operator T is given by the matrix 








1 




T = 


-y/3/2 - 


-1/2 






_ V3/2 - 


"1/2. 




The adjoint T* of the analysis operator is given by the 


matrix 


rj-iH 


-V3/2 


VS/2 




1 -1/2 


-1/2. 
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Therefore, the frame operator § is represented by the matrix 



rj-in rj-i 



-y/3/2 \/3/2 

1 -1/2 -1/2 



1 

-y/3/2 -1/2 
_ V^/2 -1/2 



1 
1 



and hence § = AI^2 with A = 3/2, which implies, by Theorem 3.20, that {gi,g2,g3} is a tight frame 
(for R 2 ). 

The design of tight frames is challenging in general. It is hence interesting to devise simple 
systematic methods for obtaining tight frames. The following theorem shows how we can obtain a 
tight frame from a given general frame. 

Theorem 3.22. Let {gk}keK. be a frame for the Hilbert space % with frame operator §. Denote the 
positive definite square root o/S _1 by S -1 / 2 . Then \^>~ l l 2 gk}keK, is a tight frame for H. with frame 
bound A = 1, i.e., 



' 1/2 g k )§- 1/2 gk, for all x en. 

it has, by Lemma 



3.12 



3.10 



a unique 



Proof. Since S _1 is self-adjoint and positive definite by Theorem 
self-adjoint positive definite square root §>~ 1//2 that commutes with S _1 . Moreover S -1 / 2 also commutes 
with §, which can be seen as follows: 

S-VSg-i = g-ig-i/2 

ss- 1 / 2 ^- 1 = §~ 1/2 

SS-V2 = §-l/2§_ 

The proof is then effected by noting the following: 

x = S _1 Sa; = §-i/2§-i/2 §;c 
= §- 1/2 §§- 1/2 x 



£(s- 1/2 *.**)s- 1/a ** 

keic 



keK. 
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□ 



It is evident that every ONB is a tight frame with A = 1. Note, however, that conversely a 
tight frame (even with A = 1) need not be an orthonormal or orthogonal basis, as can be seen from 
Example 3.4 However, as the next theorem shows, a tight frame with A = 1 and \\gk\\ = lj for 



all k G K,, is necessarily an ONB. 

Theorem 3.23. A tight frame {gk}keK. f° r the Hilbert space T~L with A = 1 and \\gk\ 
is an ONB for % . 



1, for all k G /C, 



Proof. Combining 



with 



(§gk,gk) 



(§gk,gk) = A\\gk\\ 2 
\(gk,gj)\ 2 



\9k\ 



\(9k,9j)\' 



we obtain 



\9k\\ 4 + Yl \(9k,9j)\' 



\9k\ 



Since ||<7jfc|| 2 = 1, for all k G K, it follows that Ylj^k \(9k,9j)\ 2 '■ 
elements of {gj}jeic are necessarily orthogonal to each other. 



0, for all k G /C. This implies that the 

□ 



There is an elegant result that tells us that every tight frame with frame bound A = 1 can be 
realized as an orthogonal projection of an ONB from a space with larger dimension. This result is 
known as Naimark's theorem. Here we state the finite-dimensional version of this theorem, for the 
infinite-dimensional version see |24l. 



Theorem 3.24 (Naimark, (24j Prop. 1.1]). Let N > M. Suppose that the set {gi, . . . , g N }, g k G 
H, k = 1, . . . , N, is a tight frame for an M -dimensional Hilbert space H with frame bound A = 1. 
Then, there exists an N -dimensional Hilbert space JC D H and an ONB {ei, . . . ,e7v} for /C such that 
Pefe = gfc, k = 1, . . . , N, where P : 1C — > 1C is the orthogonal projection onto H. 

We omit the proof and illustrate the theorem by an example instead. 

Example 3.25. Consider the Hilbert space K, = M 3 , and assume that H. C K. is the plane spanned by 
the vectors [1 0] T and [0 1 0] T , i.e., 

H = span 



{[10 0] T ,[0 10] T }. 

We can construct a tight frame for % with three elements and frame bound A 



1 if we rescale the 

Mercedes-Benz frame from Example 3.21 Specifically, consider the vectors gfc, k = 1,2,3, defined in 
(54) and let g' fc = y^gfc, k = 1, 2, 3. In the following, we think about the two-dimensional vectors 



l' k as being embedded into the three-dimensional space K, with the third coordinate (in the standard 
consider the following three vectors in /C: 



basis of K.) being equal to zero. Clearly, {g' fc }| =1 is a tight frame for H with frame bound A = 1. Now 










"-1/V2" 




" 1/V2- 


ei = 




, e 2 = 


-1/V6 


, e 3 = 


-1/V6 




-l/>/3 




-1/V3 




-1/V3 
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Direct calculation reveals that {e/%}| =1 is an ONB for /C. Observe that the frame vectors gl, k = 1, 2, 3, 
can be obtained from the ONB vectors e^, k = 1,2, 3, by applying the orthogonal projection from K, 
onto H: 

"1 0" 

10, 




according to g' k = Pe^, k = 1,2,3. This illustrates Naimark's theorem. 
3.5 Exact Frames and Biorthonormality 



2.2 



we studied expansions of signals in C into (not necessarily orthogonal) bases. The 



In Section 

main results we established in this context can be summarized as follows: 

1. The number of vectors in a basis is always equal to the dimension of the Hilbert space under 
consideration. Every set of vectors that spans C M and has more than M vectors is necessarily 
redundant, i.e., the vectors in this set are linearly dependent. Removal of an arbitrary vector 
from a basis for C leaves a set that no longer spans C M . 

2. For a given basis {&k]^=i evei T signal x E C has a unique representation according to 

M 

x = ^(x,e fc )e fc . (55) 

k=l 



The basis {ek}jf = i and its dual basis {ek}jf =1 satisfy the biorthonormality relation (20). 



The theory of ONBs in infinite-dimensional spaces is well-developed. In this section, we ask how 
the concept of general (i.e., not necessarily orthogonal) bases can be extended to infinite-dimensional 
spaces. Clearly, in the infinite-dimensional case, we can not simply say that the number of elements 
in a basis must be equal to the dimension of the Hilbert space. However, we can use the property 
that removing an element from a basis, leaves us with an incomplete set of vectors to motivate the 
following definition. 

Definition 3.26. Let {gk}keic be a frame for the Hilbert space H. We call the frame {gk}keic exact 
if, for all m G /C, the set {gk}k^=m is incomplete for T~L; we call the frame {gk}keic inexact if there is at 
least one element g m that can be removed from the frame, so that the set {gk}k^m is again a frame 
for U. 

There are two more properties of general bases in finite-dimensional spaces that carry over to the 



infinite-dimensional case, namely uniqueness of representation in the sense of ( 55 ) and biorthonormality 
between the frame and its canonical dual. To show that representation of a signal in an exact frame is 
unique and that an exact frame is biorthonormal to its canonical dual frame, we will need the following 
two lemmas. 

Let {gk}keK an d {gk}k^K, De canonical dual frames. The first lemma below states that for a 
fixed x G T{, among all possible expansion coefficient sequences {ck}keK satisfying x = X^fceAC c k9k, the 
coefficients Ck = (x,gk) have minimum Z 2 -norm. 

Lemma 3.27 ([5]). Let {gk}keic be a frame for the Hilbert space % and {gk}k&K its canonical dual 
frame. For a fixed x 6 T~L, let Ck = (x,gk) so that x = Ylkeic c k9k- If it is possible to find scalars 
{ofcjfcex: ¥= { c fc}fce/c such that x = Y^keK, a k9k, then we must have 

^2\ak\ 2 = J ^2\ck\ 2 + ^\ck ~ a k \ 2 ■ (56) 
keK, keK, keic 
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Proof. We have 



Cfc = (x,g k ) = (x,S l g k ) = (§ 1 x,g k ) = (x,g k ) 



with x = § 1 x. Therefore, 



(x,x) = ( Y 



and 



c k g k ,x 

keK I keK 



Y C k (9k, X) = Y c k4 = Y 



c kC k = \ c k\ 
keK keK 



(x, x) = (^2 a k9k,x ) = Y ak (g k , x) = Y a k c k- 
W/c I keK keK 

We can therefore conclude that 

2 



Hence, 



Y i cfc i 2 = Y akC *k = Yl a k°k- 

keK keK keK 

Y i Cfe i 2 + Y\ Ck ~ a ^ 2 = i Cfe ' 2 + Y( Ck ~ a ^ ( c *k ~ a k) 

keK keK keK keK 

= Y l Cfc ' 2 + l Cfc ' 2 ~ Yj Ck °*k ~ Y C *k ak + Y ' afc l 

keK keK keK keK keK 



(57) 



Using (57), we get 



Y i cfe i 2 + Y\° k ~ afc i 2 



keK 



keK 



Y\ ak \ 

keK 



□ 



Note that this lemma implies YlkeK \ a k\ 2 > SfceJt l c fc| 2 > i- e -> the coefficient sequence {afcjfcgic h&s 
larger / 2 -norm than the coefficient sequence {c k = (x,g k )} k& tc- 

Lemma 3.28 ([5]). Let {g k } k ^fc be a frame for the Hilbert space % and {{jk}keK its canonical dual 
frame. Then for each m G K, we have 

~ v ,2 1 - \(g m ,9m)\ 2 - |l - (gm,g m }\ 2 

2_^\\9m,9k)\ = 2 ■ 

Proof. We can represent g m in two different ways. Obviously g m = YlkeK a k9k with a m = 1 and a k = 
for 7^ m, so that YlkeK \ a k\ 2 = 1- Furthermore, we can write g m = YlkeK°k9k with c k = (g m ,gk}- 
From (56) it then follows that 

i = Y i afc ' 2 = i Cfc ' 2 + Y\° k ~ afc ' 2 



fce/c 



\ck\ 2 + ic. 



|2 



fee/c 



+ Y i cfc ~ afc i 2 

Y \(9m,9k}\ 2 + \ {g m ,g m ) - 1| 2 + Y \(9m,9k)\' 

k^m 



keK 



2 Y \(9m,gk}\ 2 + \(gm,g m }\ 2 + 11 - (s 



and hence 



X] \ (9m,9k) 
k^m 



l-\(9m,9m)\ -\l-(9m,9n 
2 



□ 
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We are now able to formulate an equivalent condition for a frame to be exact. 

Theorem 3.29 (|5|). Let {gk}keic be a frame for the Hilbert space T~L and {(jk}keK its canonical dual 
frame. Then, 

1- {9k}keic is exact if and only if {g m ,g m ) = 1 for all m G K; 

2- {9k}keK i s inexact if and only if there exists at least one m £ JC such that {gm,g m ) ^ 1. 

Proof. We first show that if (g m ,g m ) = 1 for all m E fC, then {gk}k^m is incomplete for T~L (for all 
mg)C) and hence {gk}k&K is an exact frame for T-L. Indeed, fix an arbitrary m G fC. From Lemma [3.28 
we have 

^ \ (g g k )\ 2 = 1 ~ \(9m,g m )\ 2 ~ |1 ~ (ffm,ffm)| 2 
ky^m 

Since {g m ,g m ) = 1, we have J^k^m \{9rn,9k}\ 2 = so that {g m ,h) = {gm,9k) = for all k ^ m. 
But g m / since (g m ,gm) = 1- Therefore, {gfcjfc^m is incomplete for because g m / is orthogonal 
to all elements of the set {gk}k^=m- 

Next, we show that if there exists at least one m G K, such that (g m ,g m ) ^ 1, then is 
inexact. More specifically, we will show that {gk}k^m is still a frame for H if {g m ,g m ) ^ 1. We start 
by noting that 

ffm = (gm,gk)gk = (gm,gm)g m + (g m ,gk)gk- (58) 

fcg/C ky^m 



If (g m ,g m ) 7^ 1, (58) can be rewritten as 



5m 



ky^m 



and for every x £ H we have 



i(z,5m)r 



1 



< 



i - (gm,g m ) 
1 

|i - {g m ,g m ) 



Y (g m ,gk) {x,gk) 

kyi-ra 



^2 \(9m,h)\ X 
ky^m 



\( x >9k)\' 

kyi-m 



Therefore 



\( x '9k}\ 2 = \(x,g m )\ 2 + \( x >9k)Y 

kaK ky^m 



< 



1 



|1 - (9m,gm)\' 

Y \{x,gk)Y 

ky^m 



Y \ (9m,h) 
ky^m 



1 + 



1 



|1 (ffmjSm)! 



Y \( x >9k)\ 2 

ky^m 

\{g m ,9k)\' 

kj^m 



+ Y \ ( x >9k) 

ky^ra 



Cj2\^9k)[ 

ky^m 
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or equivalently 

77 \( x >9k)\ 2 < E \( x >9k)\ 2 ■ 



C 



With Q36p it follows that 
A u ll2 

k^K. k^m k^K. 



where A and B are the frame bounds of the frame {gk}keic- Note that (trivially) C > 0; moreover C < 00 
since {g m ,g m ) / 1 and Kffm, h)\ < 00 as a consequence of {gk}keK being a frame for "H. This 



implies that A/C > 0, and, therefore, (59) shows that {gk}k^m is a frame with frame bounds A/C 
and B. 

To see that, conversely, exactness of {gt\ kaK, implies that {g-mj g m ) = 1 for all m E /C, we suppose 
that {gk}ketc is exact and {g m ,g m ) 7^ 1 for at least one m E JC. But the condition (g m ,g m ) 7^ 1 for at 
least one m E fC implies that {gk}keK. is inexact, which results in a contradiction. It remains to show 
that {gk}keK inexact implies (g m ,gm) 7^ 1 for at least one m E K,. Suppose that {gk}keK is inexact 
and {g m ,g m ) = 1 for all m E JC. But the condition (g m ,g m ) = 1 for all m E /C implies that {gk}keic is 
exact, which again results in a contradiction. □ 

Now we are ready to state the two main results of this section. The first result generalizes the 
biorthonormality relation (|20|) to the infinite-dimensional setting. 



Corollary 3.30 ([H]). Let {gk}keK, be a frame for the Hilbert space %. If {gk}keK ^ exact, then 
{gk}keK, and Us canonical dual {{jk}k&c are biorthonormal, i.e., 



,9k) 



1 , if k = m 
0, if k ^ m. 



Conversely, if {gk}k&K an d {9k}keK are biorthonormal, then {gk}keK ^ s exact. 



Proof. If {gk}k£K is exact, then biorthonormal 
{gm,g m } — 1 for all m E K, and Lemma 3.28 



ity follows by noting that Theorem 3.29 implies 
implies J2k^m \(9m,9k)\ 2 = for all m E K and 
thus (g m , gk) = for all k ^ m. To show that, conversely, biorthonormality of {gk}k&K, and {fffcjfceA: im- 



plies that the frame {gk}keic is exact, we simply note that (g m , g m ) = 1 for all m E )C, by Theorem 3.29 



implies that {gk}ke/c is exact. □ 

The second main result in this section states that the expansion into an exact frame is unique and, 
therefore, the concept of an exact frame generalizes that of a basis to infinite-dimensional spaces. 

Theorem 3.31 (|5|). If {g k }keK * s an exact frame for the Hilbert space T~L and x = Y2k£ic c k9k 
with x E H, then the coefficients {ck}keK. are unique and are given by 

Ck = (x,9k) , 

where {gk}k&ic * s the canonical dual frame to {gk}keic- 



Proof. We know from (50) that x can be written as x = ^fcefc i x ^9k) 9k- Now assume that there is 



another set of coefficients {ck}k&/c such that 



^c k g k . (60) 

keK. 
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Taking the inner product of both sides of ( 60 ) with g m and using the biorthonormality relation 

(9k, 9m) = 
we obtain 

(x,9m) = Ck (9k, 9m) = Cm- 
keK. 

Thus, c m = (x,g m ) for all m G K and the proof is completed. □ 

4 The Sampling Theorem 

We now discuss one of the most important results in signal processing — the sampling theorem. We 
will then show how the sampling theorem can be interpreted as a frame decomposition. 

Consider a signal x(t) in the space of square-integrable functions C 2 . In general, we can not expect 
this signal to be uniquely specified by its samples {x(kT)}k e z, where T is the sampling period. The 
sampling theorem tells us, however, that if a signal is strictly bandlimited, i.e., its Fourier transform 
vanishes outside a certain finite interval, and if T is chosen small enough (relative to the signal's 
bandwidth), then the samples {x(kT)}k£z do uniquely specify the signal and we can reconstruct x(t) 
from {x(kT)}k<=z perfectly. The process of obtaining the samples {x(kT)}k^z from the continuous-time 
signal x(t) is called A/D conversion^] the process of reconstruction of the signal x(t) from its samples 
is called digital-to-analog (D/A) conversion. We shall now formally state and prove the sampling 
theorem. 

Let x(f) denote the Fourier transform of the signal x(t), i.e., 

/•oo 

x(f)= / x(t)e- i2ntf dt. 



We say that x(t) is bandlimited to BHz if x(f) = for |/| > B. Note that this implies that the total 
bandwidth of x(t), counting positive and negative frequencies, is IB. The Hilbert space of C 2 functions 
that are bandlimited to BHz is denoted as C 2 (B). 

Next, consider the sequence of samples [x[k] = x(A;T)} fcgZ of the signal x(t) G £ 2 {B) and compute 
its discrete-time Fourier transform (DTFT): 



Uf) = E x[k]e- i2 * k f 

k=—oo 
oo 

= x(kT)e- i27Tkf 

k=—oo 
k=— oo 



where in the last step we used the Poisson summation formula^ [25[ Cor. 2.6]. 

We can see that x~d(f) is simply a periodized version of x(f). Now, it follows that for 1/T > 2B 
there is no overlap between the shifted replica of x(f /T), whereas for 1/T < 2B, we do get the different 



5 Strictly speaking A/D conversion also involves quantization of the samples. 

6 Let x{i) £ C 2 with Fourier transform x(f) = x(t)e~ l2nt f dt. The Poisson summation formula states that 
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—B B I 

(a) 

^ X d (f) 




Figure 5: Sampling of a signal that is band-limited to BHz: (a) spectrum of the original signal; (b) 
spectrum of the sampled signal for 1/T > 2B; (c) spectrum of the sampled signal for 1/T < 2B, where 
aliasing occurs. 
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shifted versions to overlap (see Figure [5]). We can therefore conclude that for 1/T > 2B, x(f) can 
be recovered exactly from x~d{f) by means of applying an ideal lowpass filter with gain T and cutoff 
frequency BT to Xd(f)- Specifically, we find that 

x(f/T) = x d (f)Th LP (f) (62) 

with 

— f 1, l/l < BT 

htp(f)= ' . (63) 

0, otherwise. 



From (62), using ( |61| ), we immediately see that we can recover the Fourier transform of x(t) from the 
sequence of samples {x[fc]}fc e g according to 

oo 

x(f)=Th LP (fT) x[k]e~ i2 ^ T . (64) 

k=— oo 

We can therefore recover x(t) as follows: 



x(t) = / x(f)e i2 ^df 

J — OO 

/OO °° 
Th LP {fT) ^2 x[k]e- i2nkfT e i2nft df 

■°° fc=-oo 

°° /'OO 

= £ / h^{fTy 2 - fT{tlT ~ k) d{fT) 

k=-oo J -°° 

fc=-oo ^ ' 

oo 

= 25T ^ x[fc] smc(2B(t - kT)), 



(65) 



fc=— oo 



where h\jp{t) is the inverse Fourier transform of /ilp(/), i-e, 

/oo 
-oo 

and 

. A sin(7rx) 

smc(x) = . 

irx 

Summarizing our findings, we obtain the following theorem. 



Theorem 4.1 (Sampling theorem 26, Sec. 7.2]). Let x(t) E £ (B). Then x(t) is uniquely specified 
by its samples x(kT), k E Z, if 1/T > 2B. Specifically, we can reconstruct x(t) from x(kT), k E Z, 
according to 

oo 

x(t) = 2BT x(kT)smc(2B(t-kT)). (66) 

fc=— oo 
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4.1 Sampling Theorem as a Frame Expansion 



We shall next show how the representation (66) can be interpreted as a frame expansion. The 
samples x(kT) can be written as the inner product of the signal x(t) with the functions 

g k (t) = 2Bsmc(2B (t-kT)), k G Z. (67) 

Indeed, using the fact that the signal x(t) is band-limited to B Hz, we get 

x(kT)= [ B x(fy 2 ^ T df = (x,g k ), 

J-B 

where 

Uf)= U~^f T ,\f\<B 
I 0, otherwise 



is the Fourier transform of g k (t) . We can thus rewrite ( 66 ) as 

oo 

x (t) = T ^2 ( x >9k) 9k{t). 

k=—oo 

Therefore, the interpolation of an analog signal from its samples {x(kT)} ke % can be interpreted as the 
reconstruction of x(t) from its expansion coefficients x(kT) = (x,<7&) in the function set {g k (t)} ke %. 
We shall next prove that {g k {t)} k <=.z is a frame for the space C 2 (B). Simply note that for x(t) G C 2 {B), 
we have 

I oo \ oo 

\\x\\ 2 = {x,x) = (T ^ ( x '9k) 9k(t),x) =T ^ \( x >9k)\ 2 

\ k=—oo I k=—oo 

and therefore 

1 ,, ,, 2 



T »*\r '= Yl \( x >9k)\ 2 - 

k=—oo 

This shows that {gk(t)}k^z is, in fact, a tight frame for C 2 {B) with frame bound A = 1/T. We 
emphasize that the frame is tight irrespective of the sampling rate (of course, as long as 1/T > 2B). 
The analysis operator corresponding to this frame is given by T : C 2 (B) — > I 2 as 

T : x ->■ {{x,g k )}kez, (68) 

i.e., T maps the signal x(t) to the sequence of samples {x(kT)}k & %. 

The action of the adjoint of the analysis operator T* : I 2 — > C 2 (B) is to perform interpolation 
according to 

oo 

c k g k . 

k=—oo 

The frame operator S : C 2 (B) — > C 2 {B) is given by S = T*T and acts as follows 

oo 

( x ,9k)gk(t). 

k=—oo 

Since {g k (t)} k& z is a tight frame for C 2 (B) with frame bound A = 1/T, as already shown, it follows 
that S = (1/T)I £2(B) . 
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The canonical dual frame can be computed easily by applying the inverse of the frame operator to 
the frame functions {g k (t)} k& z according to 



g k (t) = S~ l g k (t) = Tl C 2 (B) g k {t) = Tg k (t), G Z. 

Recall that exact frames have a minimality property in the following sense: If we remove anyone 
element from an exact frame, the resulting set will be incomplete. In the case of sampling, we have an 
analogous situation: In the proof of the sampling theorem we saw that if we sample at a rate smaller 
than the critical sampling rate 1/T = 2B, we cannot recover the signal x(t) from its samples {x(kT)} ke %. 



In other words, the set {g k {t)}k& in (67) is not complete for C 2 (B) when 1/T < 2B. This suggests 
that critical sampling 1/T = 2B could implement an exact frame decomposition. We show now that 
this is, indeed, the case. Simply note that 



(gk,gk) =T(g k ,g k ) 



\9k\ 



T \\9k\ 



2BT, for all k G Z. 



For critical sampling 2BT = 1 and, hence, (g k ,g k ) = 1, for all k G Z. Theorem 3.29 therefore allows 
us to conclude that {g k (t)} k ez is an exact frame for C 2 (B). 

Next, we show that {g k (t)} k £% is not only an exact frame, but, when properly normalized, even 
an ONB for C 2 (B), a fact well-known in sampling theory. To this end, we first renormalize the frame 
functions g k (t) according to 

g' k (t) = Vfg k (t) 



so that 



X< J) = ( x ^' k ) g' k {t)- 



k=—oo 
2/ 



We see that {g' k (t)} k ez is a tight frame for C (B) with A = 1. Moreover, we have 



llflilr = r||sfc| 



2BT. 



w k \ 



1, for all k G Z, and Theorem 3.23 allows us to conclude 



Thus, in the case of critical sampling, 
that {g' k (t)} keZ is an ONB for C 2 {B). 

In contrast to exact frames, inexact frames are redundant, in the sense that there is at least one 
element that can be removed with the resulting set still being complete. The situation is similar in the 
oversampled case, i.e., when the sampling rate satisfies 1/T > 2B. In this case, we collect more samples 
than actually needed for perfect reconstruction of x(t) from its samples. This suggests that {g k {t)} k ^z 
could be an inexact frame for C 2 (B) in the oversampled case. Indeed, according to Theorem 3.29 the 
condition 

{g m ,g m ) =2BT< 1, forallmeZ, (69) 

implies that the frame {g k (t)} k ^z is inexact for 1/T > 2B. In fact, as can be seen from the proof of 

is complete for C 2 {B). 



Theorem 



3.29 



(69) guarantees even more: for every m G Z, the set {g k (t)} kj t r 
Hence, the removal of any sample x(mT) from the set of samples {x(kT)} k ^x still leaves us with a 
frame decomposition so that x(t) can, in theory, be recovered from the samples {x(kT)} k ^ m . The 
resulting frame {g k (t)}k^=m will, however, no longer be tight, which makes the computation of the 
canonical dual frame complicated, in general. 



4.2 Design Freedom in Oversampled A/D Conversion 

In the critically sampled case, 1/T = 2B, the ideal lowpass filter of bandwidth BT with the transfer 



function specified in (63) is the only filter that provides perfect reconstruction of the spectrum x(f) 
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-1 -BT BT 1 

Figure 6: Reconstruction filter in the critically sampled case. 




Figure 7: Freedom in the design of the reconstruction filter. 



of x(t) according to (62) (see Figure [6]). In the oversampled case, there is, in general, an infinite number 
of reconstruction filters that provide perfect reconstruction. The only requirement the reconstruction 
filter has to satisfy is that its transfer function be constant within the frequency range —BT < f < BT 
(see Figure [7]). Therefore, in the oversampled case one has more freedom in designing the reconstruction 
filter. In A/D converter practice this design freedom is exploited to design reconstruction filters with 
desirable filter characteristics, like, e.g., rolloff in the transfer function. 



Specifically, repeating the steps leading from (62) to (65), we see that 



x(t)= f; X [k]h(±-k) 



(70) 



where the Fourier transform of h(t) is given by 



Hf) = 



1, l/l < BT 

arb(/), BT<\f\<\ 

0, |/| > \ 



(71) 



Here and in what follows arb(-) denotes an arbitrary bounded function. In other words, every set 
{h{t/T — k)}h£z with the Fourier transform of h(t) satisfying (71) is a valid dual frame for the frame 
{gk(t) = 2B sinc(2B(t — kT))}^^- Obviously, there are infinitely many dual frames in the oversampled 



case. 



We next show how the freedom in the design of the reconstruction filter with transfer function 



specified in (71 ) can be interpreted in terms of the freedom in choosing the left-inverse L of the analysis 



operator T as discussed in Section 3.3 Recall the parametrization (52) of all left-inverses of the 
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hout(f) 




■TZ(T) ± - 



Figure 8: The reconstruction filter as a parametrized left-inverse of the analysis operator. 



(72) 

I 2 is the orthogonal projection 



operator T: 

L = f*P + M(Ija -P), 
where M : I 2 -> U is an arbitrary bounded linear operator and P : I 

operator onto the range space of T. In (61) we saw that the DTFlj^j of the sequence [x[k] = x(kT)} k( _ 
is compactly supported on the frequency interval [—BT,BT] (see Figure [8]). In other words, the range 
space of the analysis operator T defined in (68) is the space of Z 2 -sequences with DTFT supported on 
the interval [—BT, BT] (see Figure [8]). It is left as an exercise to the reader to verify (see Exercise 6.7), 
using Parseval's theorem^] that the orthogonal complement of the range space of T is the space of 
Z 2 -sequences with DTFT supported on the set [-1/2, —BT] U [BT, 1/2] (see Figure^. Thus, in the 
case of oversampled A/D conversion, the operator P : I 2 — > I 2 is the orthogonal projection operator 
onto the subspace of Z 2 -sequences with DTFT supported on the interval [— BT, BT]; the operator 
(Ij2 — P) : I 2 — > I 2 is the orthogonal projection operator onto the subspace of ^-sequences with DTFT 
supported on the set [-1/2, —BT] U [BT, 1/2]. 

To see the parallels between (70) and ( [72] ) , let us decompose h(t) as follows (see Figure [8]) 



h(t) = h LP (t) + h aat (t) 



where the Fourier transform of hip{t) is given by (63) and the Fourier transform of h ou t(t) is 

h out (f) = <f arb(/) ' BT<\f\<l 
0, otherwise. 



(73) 



(74) 



Now it is clear, and it is left to the reader to verify formally (see Exercise 6.8), that the operator 
A : I 2 -> C 2 (B) defined as 

A : {c k } keZ -)• ]P c k h LP (^ - kj (75) 



acts by first projecting the sequence {c k }kez onto the subspace of Z 2 -sequences with DTFT supported 
on the interval [—BT, BT] and then performs interpolation using the canonical dual frame elements 



7 The DTFT is a periodic function with period one. From here on, we consider the DTFT as a function supported on 
its fundamental period [—1/2, 1/2]. 



Let {a k } ke z,{b k } ke z € I 2 with DTFT a(f) = IX-oo a * e ' an d Hf) = Er=-ocA e " > respectively. 



Parseval's theorem states that X^^L-oo a kb k = f_ 1 / 2 ®(f)b*(f)df- In particular, XfeL 



\a k \ 



= Il[%\a(f)\ 2 df. 
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9k(t) = hLp(t/T — k). In other words A = T*P. Similarly, it is left to the reader to verify formally 
(see Exercise 6.8), that the operator B : I 2 — > C 2 {B) defined as 



: {c/jfcez -> 5Z c k h out[ 7f> - fc) (76) 



fc=— oo 



can be written as B = M(Ip — P). Here, (Ip — P) : I 2 — > I 2 is the projection operator onto the subspace 
of Z 2 -sequences with DTFT supported on the set [-1/2, —BT] U [BT, 1/2]; the operator M : Z 2 -> £ 2 is 
defined as 

M : {c fc } fceZ -> 53 Cfc/iM ( = - fe J (77) 

fc=-oo ^ ' 

with the Fourier transform of tiM^t) given by 

I U, otherwise, 

where arb2(/) is an arbitrary bounded function that equals arb(/) for ST < |/| < |- To summarize, 



we note that the operator B corresponds to the second term on the right-hand side of ( 72 ) 



We can therefore write the decomposition (|70|) as 

/c=— oo 



CXJ y 

(t)= 53 

oo / i \ °° 

53 x[fc]/i L p^--fcJ + 53 a[fc]/i 



fc— — oo k=—oo 



if"* 



T*PTa:(t) M(I, 2 -P)Tx(t) 

= LTs(t). 

4.3 Noise Reduction in Oversampled A/D Conversion 

Consider again the bandlimited signal x(t) G C 2 (B). Assume, as before, that the signal is sampled at 
a rate 1/T > 2£>. Now assume that the corresponding samples x[k] = x(kT), k € Z, are subject to 
noise, i.e., we observe 

x'[/c] = x[fc] + w[fc], fceZ, 

where the w[k] are independent identically distributed zero- mean random variables, with variance 
E |w[fc]| 2 = a 2 . Assume that reconstruction is performed from the noisy samples x'[k], k 6 Z, using the 



ideal lowpass filter with transfer function /ilp(/) of bandwidth BT specified in (63), i.e., we reconstruct 
using the canonical dual frame according to 



OO / 

i — ~. \ 



T 



Obviously, the presence of noise precludes perfect reconstruction. It is, however, interesting to assess 
the impact of oversampling on the variance of the reconstruction error defined as 

a oveTsampling = \x(t) — X (t)\ , (79) 
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where the expectation is with respect to the random variables w[k], k S Z, and the right-hand side of 
(79) does not depend on t, as we shall see below. If we decompose x(t) as in (65), we see that 



"-""oversampling — \ X {^) x (0 

= E. 



(80) 



k=—oo h'=—oo 
oo 



]T w[k]h LP (^- kj 

k=-oo ^ ' 

OO OO / 4- \ 

E ^{w[k}w*[k'}}h LP ( f -k)ht P 

=— oo h'—— — ' ' 
oo 



T 



k' 



k=— oo 



/i L p ( ^ - A; 



(81) 



Next applying the Poisson summation formula (as stated in Footnote pi) to the function l(t') = 
^lp(t - e- 27 ^ with Fourier transform = h LP (-f - f) e -^/T)U+}') } we have 



;. | ,,-2„i*/ = , (t) = VJ = VJ j^,^ _ ( . )e -2»i((/T)(/+»,) (82) 



fc = — OO fc = — oo fc= — oo fc = — oo 

Since /ilp(/) is zero outside the interval — 1/2 < / < 1/2, it follows that 

oo 

£ £ LP (-/ " fc) e - 2 -(t/T)(/ + ^) = ^ Lp( _ /)e -2^/T)/ ) for ; £ [_ 1/2j 1/2] _ 
fe=— oo 



(83) 



We conclude from (82) and (83) that the DTFT of the sequence {a^ = h\jp(t/T — &)} fcgZ is given (in 
the fundamental interval / € [—1/2, 1/2]) by /ilp( — /)e -27ri (*/ T ^ and hence we can apply Parseval's 
theorem (as stated in Footnote [8]) and rewrite (81) according to 

2 



2 2 \ " 

"oversampling " / j 

k=—oo 
,1/2 



a 



a 



-1/2 
1/2 



\hM-f)e 



-27Ti(t/T)/ 



4f 



'-1/2 

a 2 2BT. 



h L p(f) 



df 



(84) 



We see that the average mean squared reconstruction error is inversely proportional to the oversampling 
factor \/{2BT). Therefore, each doubling of the oversampling factor decreases the mean squared error 
by 3dB. 

Consider now reconstruction performed using a general filter that provides perfect reconstruction 
in the noiseless case. Specifically, we have 



'(*)= £; Ak]h( t --k\ 

k=-oo K 



computed repeating the steps leading from (80) to (84) and is given by 



where h(t) is given by (73). In this case, the average mean squared reconstruction error can be 

Ld is given by 

h(f) 2 df (85) 



2 _ 2 

^oversampling ® 



1/2 



1/2 
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where h(f) is the Fourier transform of hit) and is specified in (71). Using (73), we can now decompose 
^oversampling in Q into two terms according to 



r BT 








2 


° 2 


h hP (f) 


2 dfW [ 


hoat(f) 


df. 


J-BT 




J BT<\f \<l/2 







(86) 



2BT 



We see that two components contribute to the reconstruction error. Comparing (86) to (84), we 



conclude that the first term in (86) corresponds to the error due to noise in the signal-band |/| < BT 
picked up by the ideal lowpass filter with transfer function /ilp(/)- The second term in (86) is due to 



the fact that a generalized inverse passes some of the noise in the out-of-band region BT < \ f\ < 1/2. 
The amount of additional noise in the reconstructed signal is determined by the bandwidth and the 
shape of the reconstruction filter's transfer function in the out-of-band region. In this sense, there exists 
a tradeoff between noise reduction and design freedom in oversampled A/D conversion. Practically 
desirable (or realizable) reconstruction filters (i.e., filters with rolloff) lead to additional reconstruction 
error. 



5 Important Classes of Frames 

There are two important classes of structured signal expansions that have found widespread use 
in practical applications, namely Weyl-Heisenberg (or Gabor) expansions and affine (or wavelet) 
expansions. Weyl-Heisenberg expansions provide a decomposition into time-shifted and modulated 
versions of a "window function" git). Wavelet expansions realize decompositions into time-shifted and 
dilated versions of a mother wavelet g(t). Thanks to the strong structural properties of Weyl-Heisenberg 
and wavelet expansions, there are efficient algorithms for applying the corresponding analysis and 
synthesis operators. Weyl-Heisenberg and wavelet expansions have been successfully used in signal 
detection, image representation, object recognition, and wireless communications. We shall next show 
that these signal expansions can be cast into the language of frame theory. For a detailed analysis of 
these classes of frames, we refer the interested reader to (4j. 

5.1 Weyl-Heisenberg Frames 

We start by defining a linear operator that realizes time-frequency shifts when applied to a given 
function. 

Definition 5.1. The Weyl operator W^n ■ C 2 ^ C 2 is defined as 

VfFfl : xit) -> e i2nnFt x(t - mT), 
where m, n G Z, and T > and F > are fixed time and frequency shift parameters, respectively. 

Now consider some prototype (or window) function g(t) G C 2 . Fix the parameters T > and F > 0. 
By shifting the window function g(t) in time by integer multiples of T and in frequency by integer 
multiples of F, we get a highly-structured set of functions according to 

g m ,n{t) = Wg;pg(t) = i 2 ™ Ft git -mT), m e Z, n € Z. 

The set {g m ,n(t) = e l2nnFt g{t — mT)} m&z ngZ is referred to as a Weyl-Heisenberg (WH) set and is 
denoted by {g, T, F). When the WH set [g, T, F) is a frame for C 2 , it is called a WH frame for C 2 . 
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Whether or not a WH set (g, T, F) is a frame for C 2 is, in general, difficult to answer. The answer 
depends on the window function g(t) as well as on the shift parameters T and F. Intuitively, if the 
parameters T and F are "too large" for a given window function git), the WH set (g,T,F) cannot be 
a frame for C 2 . This is because a WH set (g,T,F) with "large" parameters T and F "leaves holes in 
the time-frequency plane" or equivalently in the Hilbert space C 2 . Indeed, this intuition is correct and 
the following fundamental result formalizes it: 

Theorem 5.2 (|2lJ Thm. 8.3.1]). Let g(t) G C 2 and T,F > be given. Then the following holds: 

• IfTF > 1, then (g,T,F) is not a frame for C 2 . 

• If (g, T, F) is a frame for C 2 , then (g, T, F) is an exact frame if and only if TF = 1. 

We see that (g, T, F) can be a frame for C 2 only if TF < 1, i.e., when the shift parameters T and F 
are such that the grid they induce in the time-frequency plane is sufficiently dense. Whether or not 
a WH set (g, T, F) with TF < 1 is a frame for C 2 depends on the window function g(t) and on the 
values of T and F. There is an important special case where a simple answer can be given. 



Example 5.3 (Gaussian, [2l| Thm. 8.6.1]). Let T,F > and take g{t) = e~ l . Then the WH set 

wSf^t)} 

) 1 



mgZ, n£Z 

is a frame for C 2 if and only if TF < 1. 
5.2 Wavelets 

Both for wavelet frames and WH frames we deal with function sets that are obtained by letting a 
special class of parametrized operators act on a fixed function. In the case of WH frames the underlying 
operator realizes time and frequency shifts. In the case of wavelets, the generating operator realizes 
time-shifts and scaling. Specifically, we have the following definition. 

Definition 5.4. The operator Ym'n ^ : C 2 ^ C 2 is defined as 

: x{t) -> S n ' 2 x{S n t - mT), 
where m, n G Z, and T > and S > are fixed time and scaling parameters, respectively. 

Now, just as in the case of WH expansions, consider a prototype function (or mother wavelet) 
g(t) G C 2 . Fix the parameters T > and S > and consider the set of functions 

9m,n(t) = ^:n ] 9(t) = S n / 2 g(S n t - mT) , m G Z, n G Z. 

This set is referred to as a wavelet set. When the wavelet set {g m ,n{t) = S n ^ 2 g(S n t — mT)} me ^ ngZ 
with parameters T, S > is a frame for C 2 , it is called a wavelet frame. 

Similar to the case of Weyl-Heisenberg sets it is hard to say, in general, whether a given wavelet set 
forms a frame for C 2 or not. The answer depends on the window function g(t) and on the parameters 
T and S and explicit results are known only in certain cases. We conclude this section by detailing 
such a case. 

Example 5.5 (Mexican hat, |21[ Ex. 11.2.7]). Take S = 2 and consider the mother wavelet 

git) = ^l\l-t 2 )e-¥\ 
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Due to its shape, g(t) is called the Mexican hat function. It turns out that for each T < 1.97, the 
wavelet set 



is a frame for C 2 21 Ex. 11.2.7] 



6 Exercises 

Exercise 6.1 (Tight frames [27]). 

1. Prove that if K E Z \ {0}, then the set of vectors 



git 



^ e i2nk/(KM) g i27rfe(M-l)/(-K"M) 1 



0<k<KM 



is a tight frame for C A/ . Compute the frame bound. 
2. Fix Tel \ {0} and define for every 



9k(t) 



'e M l T , te[0,T] 
0, otherwise. 



Prove that {gk(t)}k& is a tight frame for £ 2 ([0,T]), the space of square integrable functions 
supported on the interval [0, T]. Compute the frame bound. 

Exercise 6.2 (discrete Fourier transform (DFT) as a signal expansion). The DFT of an M-point 
signal x[k], k = 0, . . . , M — 1, is defined as 



. AI-l 

m E « 

k=0 



.r\n\ - —= > .rlfcle [2n M k . 



Find the corresponding inverse transform and show that the DFT can be interpreted as a frame 
expansion in C M . Compute the frame bounds. Is the underlying frame special? 

Exercise 6.3 (Unitary transformation of a frame). Let {gk}keK. be a frame for the Hilbert space % 
with frame bounds A and B. Let U : % — > % be a unitary operator. Show that the set {Ugk}keK is 
again a frame for T~L and compute the corresponding frame bounds. 

Exercise 6.4 (Redundancy of a frame). Let {gfej^Li be a frame for C M with iV > M. Assume that 
the frame vectors are normalized such that ||gfc|| = 1, k = 1, . .. ,N. The ratio N/M is called the 
redundancy of the frame. 

1. Assume that {Sk}k=i is a tight frame with frame bound A. Show that A = N/M. 

2. Now assume that A and B are the frame bounds of {gk\k=v Show that A < N/M < B. 



Exercise 6.5 (Frame bounds |21|). Prove that the upper and the lower frame bound are unrelated: 
In an arbitrary Hilbert space % find a set {gk}keic with an upper frame bound B < oo but with 
the tightest lower frame bound A = 0; find another set {gk}keK with lower frame bound A > but 
with the tightest upper frame bound B = oo. Is it possible to find corresponding examples in the 
finite-dimensional space C M ? 
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Exercise 6.6 (Tight frame as an orthogonal projection of an ONB). Let {e^}-^ be an ONB for 
an iV-dimensional Hilbert space %. For M < N, let %' be an M-dimensional subspace of %. Let 
P : % — > % be the orthogonal projection onto H! . Show that {^ek} k=1 is a tight frame for W . Find 
the corresponding frame bound. 

Exercise 6.7. Consider the space of Z 2 -sequences with DTFT supported on the interval [/i,/2] with 
— 1/2 < fx < f2 < 1/2. Show that the orthogonal complement (in I 2 ) of this space is the space of 
P-sequences with DTFT supported on the set [—1/2, 1/2] \ [fx, fe]. [Hint: Use the definition of the 
orthogonal complement and apply Parseval's theorem.] 



Exercise 6.8. Refer to Section 4.2 and consider the operator A in (|75j) and the operator T in (68). 

1. Consider a sequence {a^jfeez G 7£(T) and show that A{ak}kez = ^*{(ik}kei.- 

2. Consider a sequence {bk}k& £ TiiT) 1 - and show that A{bk}k& = 0. 

3. Using the fact that every sequence {ck}kez can be decomposed as {ck}kez = {«fc}fcez + {bk}kez 
with {a k } k& G K(T) and {b k } k & G ^(T)- 1 , show that A = f*P where P : Z 2 -> Z 2 is the 
orthogonal projection operator onto 7£(T). 

[Hints: Use the fact that 7£(T) is the space of Z 2 -sequences with DTFT supported on the interval 
[-BT, BT]; use the characterization of IZ^) 1 - developed in Exercise 6.7 work in the DTFT domain.] 



Exercise 6.9. Refer to Section [4.2| and use the ideas from Exercise 6.8 to show that the operator B 
in (76) can be written as B = M(I/a — P), where P : I 2 — > I 2 is the orthogonal projection operator 



onto 1Z(Y) with T defined in (68); and M : I — >■ C is the interpolation operator defined in (77) 



Exercise 6.10 (Weyl operator [28]). Refer to Definition 5.1 and show the following properties of the 
Weyl operator. 

1. The following equality holds: 

w( T ' F )W (T ' F) - P -U™ITF W (T,F) 
w m,n w fc,« — e VV m+fc,n+r 

(rp rp\ 

2. The adjoint operator of Wm,n is given by 

-i2TrmnTF m r(T,F) 



(wL T f } )' 



" — m,— re" 



is a 

m£Z, n£Z 



3. The Weyl operator is unitary on C 2 , i.e., 

Exercise 6.11 (Dual WH frame 3 ). Assume that the WH set {g m ,n{t) = W^'f ) c/(t)| 
frame for C 2 with frame operator S. 

1. Show that the frame operator § and its inverse S _1 commute with the Weyl operators, i.e., 

W (T ' F) S = §W (T ' F) 
W (T ' F) § _1 = § _1 W (T ' F) 

vv m,n " ^ vv m,n 

for m, n G Z. 
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2. Show that the minimal dual frame {g m ,n{t) = (§ 1 9m,n)(t)} is a WH frame with prototype 
function g(t) = (§ _1 <7)(i), i.e., that 



Exercise 6.12 (WH frames in finite dimensions). This is a Matlab exercise. The point of the exercise 
is to understand what the abstract concept of the frame operator and the dual frame mean in linear 
algebra terms. 

Consider the space C M . Take M to be a large number, such that your signals resemble continuous- 
time waveforms, but small enough such that your Matlab program works. Take a prototype vector 
g = [g[l] • • • g[M]] T G C M . You can choose, for example, the firl{.) function in Matlab, or discrete 
samples of the continuous-time Gaussian waveform e~ x I 2 . Next, fix the shift parameters T, K G N in 



such a way that L = M/T G N. Now define 

9k l [n] 4 g [(n - IT) mod M] e ^ kn / K , k = 0, . . . , K - 1, I = 0, . . . , L - 1, n = 0, . . . , M - 1 



1. Show that the analysis operator T : C — > C can be viewed as a (KL x M)-dimensional 
matrix. Specify this matrix in terms of g, T, K, and M. 

2. Show that the adjoint of the analysis operator T* : C KL — > C M can be viewed as an (M x KL)- 
dimensional matrix. Specify this matrix in terms of g, T, K, and M. 

3. Specify the matrix corresponding to the frame operator S in terms of g, T, K, and M. Call this 
matrix S. Compute and store this M x M matrix in Matlab. 

4. Given the matrix S, check, if the WH system {gk,i} k=0 K _ 1 ; =0 L _ x you started from is a 
frame. Explain, how you can verify this. 

5. Prove that for K = M and T = 1 and for every prototype vector g / 0, the set {gk,i} k=0 K _ x i=q 
is a frame for C M . 

6. For the prototype vector g you have chosen, find two pairs of shift parameters (T\,K\) and 
(T21K2) such that {gfc,«} fc=0 K _i i =0 L _i is a frame for T = T\ and K = K\ and is not a 
frame for T = T2 and K = K2. For the case where {gk,l} k=0 K _ x i=0 L _i is a frame, compute 
the frame bounds. 

7. Compute the dual prototype vector g = [g[l] ■ ■ ■ g[M]] T = S _1 g. Show that the dual frame 
{Sk,l}k=o,...,K-i,i=o,...,L-i is S iven b y time-frequency shifts of g, i.e., 



£lk,i[n\ = g[(n - IT) mod M] e i2nkn/K , k = 0,...,K-l, I = 0, . . . , L - 1, n = 0,...,M-l. 



9mAt)=^;n ] 9(t)- 



and construct a discrete-time WH set according to 





with 
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Notation 



a, b, . . . vectors 
A, B, . . . matrices 

a T , A T transpose of the vector a and the matrix A 

a*, a*, A* complex conjugate of the scalar a, element- wise complex conjugate of the 

vector a, and the matrix A 
a H , A H Hermitian transpose of the vector a and the matrix A 

Ijv identity matrix of size N x N 

rank(A) rank of the matrix A 

A(A) eigenvalue of the matrix A 

A m in(A), A m m(A) smallest eigenvalue of the matrix A, smallest spectral value of the self-adjoint 
operator A 

Amax(A), A max (A) largest eigenvalue of the matrix A, largest spectral value of the self-adjoint 
operator A 

i 

= definition 
A, £>, . . . sets 

R, C,Z, N real line, complex plane, set of all integers, set of natural numbers (including 

zero) 

C 2 Hilbert space of complex-valued finite-energy functions 

C 2 {B) space of square-integrable functions bandlimited to B Hz 

H abstract Hilbert space 

I 2 Hilbert space of square-summable sequences 

(a, b) inner product of the vectors a and b: (a, b) = ^ [a], ([b],)* 

(x, y) depending on the context: inner product in the abstract Hilbert space % or 

inner product of the functions x(t) and y(t): (x,y) = x(t)y*{t)dt 
||a|| 2 squared ^ 2 -norm of the vector a: ||a|| 2 = \ [a]J 2 

||y|| 2 depending on the context: squared norm in the abstract Hilbert space H or 

squared £ -norm of the function y(t): \\y\\ 2 4 \y(t)\ 2 dt 
I^,Ij2,l£2,l£2(s) identity operator in the corresponding space 
1Z(A) range space of operator A 

A* adjoint of operator A 

x(f) Fourier transform of x(t): x(f ) = x(t)e~ l27rt -> dt 

Xd(f) Discrete-time Fourier transform of x[k]: Xd{f) — Sfel-oo x[k]e~ l2nk ^ 
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